This invention relates to a system for measuring a signal quality, and more particularly to a system for measuring quality of multimedia signals communicated via a network.
Numerous processing and transmission methods have been devised for the communication of voice, audio, video, and multimedia information. These types of signals are collectively referred to herein as xe2x80x9cvoice, audio and videoxe2x80x9d (VAV) signals. VAV signals may be processed in many different ways before being transmitted to a receiver. For example, a VAV signal may be digitized, compressed, and modulated onto a carrier. The receiver then must transform the received signal back into a perceptible representation of the VAV signal.
Traditionally, received VAV signal quality is evaluated by subjective testing. However, this type of test is not practical as an in-service testing method, nor does this type of test produce consistent, reproducible results. Signal quality evaluation is further complicated when VAV communication is incorporated into very complex transport systems such as cellular code division multiple access (CDMA) systems or wideband CDMA systems. Different, but equally difficult challenges are presented when evaluating VAV quality in such non-deterministic environments as the Internet.
It is known to evaluate signal quality indirectly by measuring parameters such as signal-to-noise ratio (S/N), carrier-to-interference ratio (C/I), lost packet rate, and bit error rate (BER). However, it is difficult to relate these parameters to user perception of the quality of received VAV signals, particularly when such signals are highly compressed or processed. Also, these parameters are not well-suited to reflect VAV quality correctly and accurately over time.
Algorithms to estimate perceptual speech quality are known. For example, International Telecommunications Union standard P.861 is an objective algorithm that can be used to automatically compute a qualitative figure such as a Mean Opinion Score (MOS) for speech transmission. However, these types of tests require both an original and a test signal to be statically analyzed by an item of test equipment. Thus, if quality evaluation is to be done in the field with real signals, the test equipment must incorporate network architecture and protocols into its hardware, software, or firmware. Often, the network architecture is not available, or it may be difficult or expensive to implement. Proprietary restrictions may even preclude its implementation.
In view of the above, methods and apparatus for evaluating quality of received VAV signals that do not require explicit knowledge of the operation and configuration of a network, such as the transport technology used and the network access technology, would be desirable. It also would be desirable if the methods and apparatus provide real-time or non-real-time automated measurements of quality, so that interactive field testing, for example, could be performed. In addition, a common test platform for various types of VAV signals that easily connects to an equipment under test would be advantageous. Furthermore, methods and apparatus that provide reliable, repeatable, and easy-to-understand quantitative quality of service measurements for one-way, multi-hop, or round-trip measurements would be desirable.
In one embodiment, the invention is a method for measuring perceptual quality of voice signals, audio signals, audio-video signals, and multimedia signals in communication equipment. A processed test signal is communicated via a network from a first device to a second device. The processed test signal is then received by an equipment under test, which further processes the processed test signal to recover a representation of the test signal. The recovered test signal is then objectively analyzed to determine a measure of perceptual quality by comparing the recovered test signal to a pre-stored representation of the test signal.
In another embodiment, the invention is a quality measurement unit that attaches to equipment under test to identify and evaluate quality of recovered test signals. The test signals are those communicated from a remote device and those communicated by the equipment under test. In the latter case, the quality measurement unit is configured to communicate with the equipment under test and the network, so that the quality measurement unit itself receives a representation of the communicated test signal by the equipment under test.
It will be seen that, in methods and apparatus of this invention, quality of received VAV signals are evaluated without an explicit knowledge of the network via which the equipment under test communicates. Moreover, both real-time and non-real-time quality analysis is possible, utilizing a common test platform for various types of VAV signals. The invention is applicable to a variety of different types of quantitative quality of service measurements, including one-way, multi-hop, and round-trip measurements.